On Using Toeplitz and Circulant Matrices for Johnson-Lindenstrauss Transforms

The Johnson-Lindenstrauss lemma is one of the corner stone results in dimensionality reduction. It says that given N , for any set of N vectors X ⊂ Rn, there exists a mapping f : X → Rm such that f(X) preserves all pairwise distances between vectors in X to within (1 ± ε) if m = O(ε lgN). Much effort has gone into developing fast embedding algorithms, with the Fast JohnsonLindenstrauss transform of Ailon and Chazelle being one of the most well-known techniques. The current fastest algorithm that yields the optimal m = O(ε lgN) dimensions has an embedding time of O(n lgn + ε lg N). An exciting approach towards improving this, due to Hinrichs and Vybíral, is to use a random m×n Toeplitz matrix for the embedding. Using Fast Fourier Transform, the embedding of a vector can then be computed in O(n lgm) time. The big question is of course whether m = O(ε lgN) dimensions suffice for this technique. If so, this would end a decades long quest to obtain faster and faster Johnson-Lindenstrauss transforms. The current best analysis of the embedding of Hinrichs and Vybíral shows that m = O(ε lg N) dimensions suffices. The main result of this paper, is a proof that this analysis unfortunately cannot be tightened any further, i.e., there exists a set of N vectors requiring m = Ω(ε lg N) for the Toeplitz approach to work.